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Abstract 

This work describes a formative assessment model for a Mathematical Analysis course taken by 
engineering students. It includes online quizzes with feedback, a portfolio with weekly assignments, 
exams involving the use of mathematical software and a project to be completed in small groups of two 
or three students. The model has been perfected since 2009, and during the 2014-15 academic year the 
creation of a pilot online learning community was added. Based on Google+, it has been used for a 
peer assessment experiment involving student projects, among other uses. 
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1. Introduction 

Adaptation to the European Higher Education Area presents a model of learning that is both student 
focused and competence based, which implies new rules for assessment. Competence is defined as “...a 
roughly specialized system of abilities, proficiencies, or skills that are necessary to reach a specific 
goal...” (Weinert, 2001). Along with specific competences in the subject matter, the degree programs 
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include generic or multidisciplinary competences and thus the faculty is obliged to design 
activities that enable them to be developed and assessed. A good assessment model is essential 
for effective learning and must consider, along with the summative aspect (needed to assign a 
grade), a formative component that relies on feedback strategies, which may be assisted by 
technology. 

The mathematics courses in the engineering degree programs tend to have extensive syllabi, 
which makes it difficult to manage class time and the students' work. Many professors, arguing a 
lack of time, are reluctant to innovate and work on competences. In the study conducted by 
Garda, Garcia, Lias, Mahillo, Minano and Pinero (2013) it was shown that a great majority of the 
mathematics courses in the IT degree programs at Spanish universities propose to assess generic 
competences with the same activities used to assess mathematics. This can be a valid approach, as 
long as the indicators for each competence are adequately defined and assessed. Both Diaz, 
Garcia and de la Villa (2011) and Tuning Educational Structures in Europe (2006) demonstrate 
how normal mathematical learning activities help develop multidisciplinary competences, such as 
problem solving, communications skills, analysis and synthesis, and critical thinking. Each of 
these competences is related to one or more of the mathematical competences defined in the 
KOM Project (Niss & Hojgaard, 2011) (Figure 1). Based on this relationship, it is possible to 
design activities to develop and assess both types of competences simultaneously. 



Figure 1. Mathematical competences (KOM Project) 
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In the aforementioned study by Garda et al. (2013) on the models of assessment for the 
mathematics courses in the Computer Engineering degree programs, it was also observed that 
many courses incorporate the use of a mathematical software package for teaching and learning 
and in some way assess the exercises completed by the students, although few of them allow the 
use of software on exams. Garcia, Garcia, Martin, Rodriguez and de la Villa (2014) demonstrate 
the usefulness of integrating mathematical software in the different assessment activities. 
Moreover, the benefits of online assessment have also been identified in numerous works. 
Trenholm (2007) highlights the importance of its formative aspect, while Limniou and Smith 
(2014) experimentally verify the improvement resulting from the dynamic feedback and the 
possibility to make new attempts. Gikandi, Morrow and Davis (2011) present a report that 
indicates the potential of technology to: 

• Help clarify learning objectives and assessment criteria. 

• Facilitate the development of self-assessment activities. 

• Provide students with quality information and immediate feedback. 

• Facilitate communication between students and professors. 

• Offer the opportunity to correct deficiencies in previous instruction. 

• Facilitate data collection and processing for instructors. 

The aim of this article is to show how the introduction of technology in a continuous assessment 
model facilitates the work of the instructor and favors the formative aspect of the assessment, 
thanks to quick feedback that promotes continuous improvement. 

We present the assessment model used in the Mathematical Analysis course in the Higher 
Technical School of Computer Systems Engineering at the Technical University of Madrid. This 
continuous assessment model has been under development since the 2009-10 academic year, with 
the objective of fostering significant learning and assessing both specific mathematical and 
generic competences. 

The article is organized as follows: first, the model is presented, describing the characteristics of 
the course and the different types of activities used for assessment, as well as the changes that 
have been introduced as the result of the experience gained. Next, some data from the last 
completed year (2014-15) are discussed. Finally, some conclusions and proposals are presented. 
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2. Description of the model 

The assessment model that we have been applying, with successive improvements since the 2009- 
10 academic year, is based on the cornerstones of formative assessment: 

• The learning objectives and grading criteria must be clear and explicit for each activity. 

• Dynamic feedback positively affects learning and promotes communication between 
instructors and students. 

As an online communication tool providing support for learning and assessment, we primarily 
use the Moodle platform, which provides (among others) the following advantages: 

• Easy access and use. 

• A variety of resources to manage instructional activities. 

• Institutional support. 

• It is widely used. 

The Mathematical Analysis course is worth 6 ECTS credits. It is taught to different groups of 
around 70 students each, which are fully coordinated with one another and have a common 
Moodle environment. The specific mathematical competences are the usual ones addressed in a 
first-year calculus course for engineering students, although an attempt is made to emphasize 
those that are more closely related to computer science. The contents are divided into eight 
topics, which are organized in three modules. The classroom contact hours are divided between 
master classes with varying degrees of participation and practical lessons (in a classroom or 
laboratory setting), in which the students work to solve problems using the mathematical free and 
open code wxMaxima software. Below is a brief description of the assessment activities, 
including an explanation of how they are applied and how they have evolved over the years. 
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2.1 Online Moodle Quizzes (MQ) 

For each of the eight topics, students are required to complete an MQ, designed by their 
professors according to the following protocol: 

• The learning results were defined and about ten indicators were established. 

• For each indicator, six or seven true/false questions were written, along with their 
corresponding feedback. 

• The MQ was drafted with one question per indicator, chosen randomly by the system 
from among the six or seven questions available. 

The MQs are a learning tool that has a certain weight on the summative grade. Once the 
theoretical aspects of a topic have been explained, the students are given a couple of days to 
complete some trial self-assessments. A specific time is then indicated for everyone to connect 
online outside the classroom and complete the MQ (different for each student). The students are 
allowed two attempts, the best of which is counted. Each MQ has a weight of 0.5% on the final 
grade, as long as at least 80% of the questions are correctly answered. The total weight of the MQ 
on the final grade is 4%. In other words, if a student earns a grade (correctly answering 80% of 
the questions) on all 8 MQs, he/she would earn 0.4 of the 10 points of the final grade. 

Over the years, as the result of the experience gained, certain improvements have been added. 
Feedback is no longer limited to merely indicating the correct answer, but rather to providing the 
justification or results on which it is based. Moreover, students receive feedback for both correct 
and incorrect answers (since it is possible to have a correct answer, but with faulty reasoning). As 
a complement, we have enlisted the students' help to create a glossary in Moodle, to provide 
quick access to the most important concepts. Last year, we limited to three the number of trial 
tests prior to each questionnaire, after observing that some students completed a large number of 
trial tests, but spent just a few minutes on each one, in an attempt to discover all the possible 
questions. 
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2.2 Portfolio 

Throughout the semester, the students complete a series of short tasks, sometimes individually, 
sometimes as a team; some in the classroom with the professor's assistance, and others outside 
the classroom. All are returned corrected. The weight of this set of activities on the final grade 
has increased over the years, and they presently account for 21% of the final grade. These tasks 
include the writing of summaries, the creation of concept maps, the solving of exercises and 
problems, an occasional group workshop assignment and several ten minutes test about basic 
concepts. During the 2012-13 academic year, the requirement was introduced to have completed 
at least 70% of these tasks in order to pass the course through continuous assessment, as it is 
believed to be the most effective way to control attendance and class participation. 


2.3 Small group project (SGP) 

Working in small groups of two or three people, the students are required to complete an 
assignment, for which they are given a choice from among several different options, including: 

• Reading a mathematical document to learn a new concept, technique or algorithm on 
their own. 

• Using wxMaxima software to implement procedures that allow them to apply the 
technique studied to solve problems. 

• Solving a specific problem assigned to the group from a collection of different problems. 

• Drafting a report with the following sections: Summary, Introduction, Description of the 
method, Implementation and examples, Solution to the problem, Conclusions and 
Technical details. 

• Uploading the report and the wxMaxima files, implementing the procedures and 
calculations performed, to the Moodle platform. 

The SGP is separate from the rest of the portfolio tasks in order to give it greater emphasis, as it 
is the first step towards the complete resolution of an engineering problem, and it enables the 
development of competences such as autonomous learning, teamwork and problem solving. 

Each student spends an average of 10 hours on this project and it is worth 10% of the final 
grade. After the successive improvements, the assessment process presently consists of: 


- 96 - 



Journal of Technology and Science Education — http://dx.doi.org/10.3926/jotse.190 


• Correcting and grading the first version of the SGPs, according to a previously posted 
rubric and returning it to the students, along with suggestions for improvement. 

• Giving students the opportunity to improve their grades with a second version of the 
assignment, in which they correct their errors and implement the proposals for 
improvement. 

• The grade for the second version is weighted by that of an individual validation test 
administered in the classroom, which requires students to solve a problem similar to that 
of the project, using the tools that were practiced. 

The two-stage correction is the product of experience. We have observed the effectiveness of 
giving a grade that the students know they can improve by a certain percentage by incorporating 
the suggested modifications. 

The validation test was introduced in 2011, when it was observed that the teams were not 
distributing the work equally. As it is completed individually, the validation test detects those 
students who have done little work and intend to take advantage of the work done by their 
teammates. Students who do not demonstrate that they have achieved the objectives have their 
SGP grade reduced drastically, while those who show that they have met them improve their 
grade. 


2.4 Mandatory guided practice 

65% of the grade is based on exams. However, practice activities have been designed to foster 
significant learning. Extensive practice exercises, with clearly specified learning objectives, as well 
as theoretical questions, exercises and problems are provided for each of the three modules of 
the course. This activity is carried out throughout the instructional period of the module. In order 
to be able to take the corresponding exam, students must hand in the fully completed practice 
exercises, but it is not this work, rather the exam that will be graded. The exam is written by 
selecting some of the objectives and including exercises similar to those completed during the 
practice activities. 
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2.5 Pilot experience creating the APLICA_MATES online community 

During the 2014-15 academic year, we created an online learning community based on Google+, 
in which student participation was voluntary. At the beginning, an open community was created 
with the aim of providing collaborative work tools, permitting the exchange of ideas and the 
communication of news and other activities, and making the students' work more relevant. But 
later, due to the dispersion of the members and the difficulties this poses when trying to 
distinguish the contributions made by our students, we redefined it as a private community. As 
an experiment, students were invited to share their SGPs on a voluntary basis, and these were 
assessed by some of their classmates. The assessment work was carried out after the semester had 
ended, and was recognized with one ECTS credit for miscellaneous activities. In this process, 
each student was assigned a SGP with a topic different from that he/she had completed, and 
they were given an assessment sheet that was similar to that used by the professors, but that 
included three questions related to an academic topic, in order to assess the assimilation of the 
concepts presented. The community has also been used to host a project competition, including 
the posting of proposals and an online vote between the two finalists. 

Finally, it should be mentioned that, in fulfillment of legal requirements, in addition to the 
continuous assessment model, an alternative assessment consisting of a single exam is also 
offered, which students can request before October 30th. This is chosen by a minority of the 
students and has very poor results. 


3. Data and results 

Some data corresponding to the 2014-15 academic year are provided below. A total of 372 
students were registered. The Mathematical Analysis course is common to the degree programs 
in Computer Engineering (CE), with 109 students, Software Engineering (SE), with 180 students, 
and Information Systems (IS), with 83 students. Of the 308 students who selected the default 
option (continuous assessment), 242 completed the minimum required tasks, and of them 143 
passed the course. It should be pointed out that the efficiency rate for this course, traditionally 
considered to be “the hardest”, is currently better than that of other courses from the same 
semester (see Table 1, which presents the data from the university's semester report). 
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CE 

SE 

IS 

Mathematical Analysis 

43.12 

37.80 

33.73 

Introduction to Computers 

14.91 

14.95 

5.68 

Programming 

42.34 

44.23 

32.14 

Discrete Mathematics 

51.46 

58.05 

30.95 


Table 1. Efficiency rates for courses common to all three 


degree programs 


With regard to the assessment model, it is well considered by the students, who expressed their 
approval on a survey to evaluate the instruction. The mean score for the assessment model used 
in the Mathematical Analysis course was 4.86 points (out of 6), while the mean for all courses 
was 4.54. 

Participation in the MQs (see Figure 2) continued, but decreased throughout the course. The 
number of students who scored points makes it possible to identify the topics that are 
conceptually the most difficult. Even though the contribution to the final grade of this activity is 
small, its impact is significant, as more than 75% of the students with success on five or more 
MQs passed the course. 



Figure 2. Results of the MQs 


The group work (SGP), which is worth 10% of the final grade, was completed by 240 students. 
The assessment guidelines do not stipulate whether it is mandatory to complete the project in 
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order to pass the course. However, only one student passed the course without having completed 
the project. The average grade for the written assignments was 6.08. After the individual 
validation test, 78 students improved their grades, while 142 students saw their grades go down. 
The SGP was passed by 116 students, of whom 91 passed the course. 

At the end of the 2014-15 academic year, the private community APLICA_MATES had 77 
members. Student participation was not very active, possibly due to the initial problems. It 
cannot be said that the pilot project was a clear success, since participation was quite a bit lower 
than in the Moodle activities. A total of 18 SGPs were uploaded, and 17 students completed the 
assigned task of assessing an SGP. The responses to the three academic content questions on the 
assessment sheet were satisfactory, which indicates that assessing the SGPs has expanded 
knowledge, in addition to generic competences. The final grades assigned by the students (see 
Figure 3) differ little from those assigned by the professors. An analysis of variance shows (with a 
p-value of 0.9) that there are no significant differences. 


Peer evaluation and Teacher evaluation: Scores comparison 

10 



■ Teacher's Marks ■ Student’s Marks 


Figure 3. Peer assessment 


In the project competition, participants shared their proposals and used a survey format to vote 
online for one of two projects in the final round; in total, 44 community members participated. 
Figure 4 shows the results. 
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Electronicvoting results 
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Figure 4. Electronic voting, completed by the APLICA_MATES community 


4. Conclusions 

Over these years of successive innovations, we have observed that increasing the demand of the 
work throughout the semester has a positive effect on the results. For this ongoing work, timely 
feedback is vitally important, but this is difficult for the professor to provide, unless tasks are 
included that can be corrected quickly and/or automatically, with the help of technological tools. 

With regard to the online learning community, we have witnessed some positive aspects: it 
facilitates peer assessment, gives relevance to the students' work and can be a suitable 
environment for certain portfolio tasks. But we have also encountered certain challenges: it is 
difficult to use different communication environments if they are not well integrated. The 
separate maintenance of the Moodle environment and the online social network complicates the 
work and forces those involved to use different tools. 

Our current challenges are: 

• To find a way to enable students to make the most of online environments, publish their 
work on open platforms, use collaborative learning tools and improve their academic and 
social integration. 

• To improve and streamline feedback processes for both students and professors. 
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Firstly, we want to achieve a better integration of the online learning community in the Moodle 
environment and make the community more relevant. As a matter of fact, the number of 
members has increased to 117 in December 2015. Furthermore, we have recently offered the 
option to participate as a guest, as long as applicants can justify their interest. 

Secondly, we are experimenting with the use of the application Socrative 
(http://www.socrarive.com/) to replace some of the tasks that are carried out in the classroom with 
questions, problems and/or short exercises that are presented by the professor and answered by 
the students using their mobile telephones. Feedback in this case is immediate (for both students 
and the professor) and the method promotes class participation and discussion. 
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